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ABSTRACT 

One of the new measures developed as part of the 
Apple Classrooms of Tomorrow (ACOT) program is described. The ACOT 
project examines the impact of access to educational technology on 
the kindergarten through grade 12 classroom environments. The new 
measure is a technology-based classroom observation instrument for 
documenting the impact of technology on classroom instruction. The 
instrument uses a time-sampling procedure and is organized for 
recording in timed intervals using a machine-scannable form. 
Observers code a few key indexing variables for the activity period 
observed as follows: (1) subject area; (2) number of students 
assigned; (3) classroom observation; (4) adult roles (directing 
instruction, facilitating instruction, management and discipline, and 
not present); (5) symbol systems serving key instructional functions; 
(6) symbol systems that students use; (7) length of responses 
expected of students; (8) level of processing expected of students; 
(9) resources in use; and (10) students 1 responses to the activities. 
The instrument is currently used in documentation of high technology 
access classrooms (a database w:th 12 total hours observed in 
language arts and 6.1 hours in mathematics) and in documentation of 
changes in instructional practices associated with technology use 
over time (observations at elementary school and secondary school 
sites). The instrument's usefulness will ultimately depend on 
coordination with other data gathering techniques. Two tables present 
study findings, and 32 bar graphs provide examples of the "snapshot" 
graphical displays generated by the instrument. A 35-item list of 
references is included. (SLD) 
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Since 1987, UCLA's Center for Technology Assessment has 
been conducting a set of evaluation, research, and develop- 
ment activities at selected Apple Classrooms of Tomorrow" 
(ACOT'm) sites, with the goal of documenting the impact of 
technology access on K-12 environments (Baker, 1988; Baker & 
Herman, 1988, 1989; Baker, Herman, & Gearhart, 1988; Baker, 
Gearhart, & Herman, 1990, 1991; Baker & Niemi, 1990, 1991; 
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Gearhart, Herman, Baker, Novak, & Whittaker, 1990; Gearhart, 
Herman, & Whittaker, 1991; Gearhart, Herman, Whittaker, & 
Novak, 1991; Herman, 1988 ) . When Eva Baker and Joan Herman 
initiated the work with ACOT in 1987, the project had been 
implemented in selected classrooms at five sites that were 
dispersed nationally and varied considerably in student char- 
acteristics and school context factors. Students and teach- 
ers in all classrooms were provided with high access to indi- 
vidual computer support both at home and at school, and 
ACOT's goal was to document how instructional innovations 
emerge in high access environments. Since 1987, the ACOT 
project has evolved to encompass more sites and has assumed a 
more directive role in the kinds of teacher support provided. 
It has shifted from a 'bottom-up' exploration of the impact 
of technology access to a research and development laboratory 
for the construction of new technology tools for instruction 
and new tools for the assessment of instruction and instruc- 
tional impact. 

The work of the UCLA Center for Technology Assessment 
has evolved as well. From the outset, the Center's goal for 
its work with ACOT has been to develop a model of technology 
assessment in K-12 environments by exploring the utility and 
applicability of existing measures and by developing new 
measures as needed. The shift in our work has been one of 
emphasis-as a result of continued confrontations with the 
limitations of existing measures, the development of new 
assessment tools has become our primary focus. 
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This paper is a description of one of our new measures, 
a technology-based classroom observation instrument for docu- 
menting the impact of technology on classroom instruction. 
In the report that follows, we explain the need for a new 
observation tool sensitive to technology impact, and then 
illustrate the utility of the tool with samples of two of our 
current approaches to data analysis. 

A Technology-Based Classroom nh«Ar Yn f, ton Tool 
Our ongoing evaluation of the ACOT project required a 
method for documenting instructional impact and for providing 
"process" explanations for student and teacher outcomes. The 
instrument we had in mind would provide fairly comprehensive 
"snapshots" of classroom activities that would reveal varia- 
tions in instructional practices related to uses of diverse 
resources. Based on data produced from informal observations 
we needed a tool that could: document subject-specific 
instructional patterns, determine whether technology limits 
certain kinds of classroom organizations and supports others, 
describe how teachers' roles may shift when technology is in 
use, document how the nature of students' work differs when 
technology resources are in use (e.g., its challenge, length, 
the media used), and determine whether technology use has an 
impact on students' responses to instruction (e.g., their 
engagement with peers, or investment in their work) . No 
existing observation instrument was available to provide us 
with data appropriate to our needs . 



The Limits of Available Observation instruments for Q 1]r 

# Purpose. 

Available observation instruments most commonly focus 
on the teacher's instructional and support roles during 

# teacher-student interactions (see reviews by Cazden, 1986; 
Dunkin & Biddle, 1974; Evertson & Green, 1986). Some inter- 
action schemes are motivated by process-product analyses of 

# common functions of classroom talk that are believed to 
influence student outcomes (e.g., Flanders, 1970; Good & 
Brophy, 1983) . These schemes vary in their explication of a 

# model of the cognitive functions of interaction in supporting 
students' learning and reflection. Other schemes are 
derived from linguistic or sociolinguistic analyses of 

# discourse (e.g., Green & Wallat, 1981; Sinclair & Coulthard, 
1975) and are used in investigations of a range of research 
questions, including the cultural context of teaching and 

f learning, the functions of language in intellectual activi- 

ties (e.g, Cazden, John, & Hymes, 1972), and the communica- 
tive requirements of classroom participation (e.g., Mehan, 

0 197 9) . Observation methods derived from both traditions 

included a considerable range of procedures: on-the-spot 
coding procedures using either time sampling or event 

9 sampling techniques, post-observation coding of video- or 

audiotape, and ethnographic examinations of selected case 
excerpts . 

6 While we viewed description of classroom interaction as 

important to our scheme, it was clear that we needed an 




instrument which could provide a more comprehensive look at 
classroom activities. The instruments that were closest in 
rationale, design, and content were those developed by 
Stallings (Stallings, 1975; Stallings & Giesen; 1974; 
Stallings & Kaskowitz, 1974), later adapted by Sirotnik 
(Giesen & Sirotnik, 1979; Sirotnik, 1979), and used for two 
highly regarded large-scale evaluations of school programs: 
A Study of Schooling (Goodlad, Sirotnik, & Overman, 197 9) and 
Follow Through (Stallings, 1975; Stallings & Kaskowitz, 
1974) . The schemes included the physical environment inven- 
tory (PEI) , daily summary (DS) , classroom snapshot (CS) , and 
five minute interaction (FMI) . The schemes varied in method- 
once per classroom for the PEI and DS vs. time and sampling 
for the CS and FMI (at four times during the day, all perti- 
nent activities [CS] or events [FMI] are coded) . Data were 
collected for each scheme at different times of the day. The 
CS captured relations among activities (subject area as well 
as instructional activity, such as demonstration, discussion, 
work on written assignments), directors (teacher, aide, 
student, group, class, independent), and group size. The FMI 
entailed event recording of classroom interactions: persons 
involved (Who, Whom), interactions (What-Adult: e.g., direct 
questions, response, imperative, encouragement, moni- 
tor/observe; Student: e.g., directive, response, refusal, 
question), context (e.g., instruction, behavior, routines), 
and means (How-e.g., touching, with humor, with materials, 
negative affect) . 



These instruments were helpful guides to our own design 
efforts. We particularly appreciated the potential flexibil- 
ity of a scheme containing multiple dimensions that can be 
cross-classified to produce a broad range of analyses. 
However, the restriction on concurrent use of the four 
schemes would limit our ability to describe as richly as 
possible any giv»n classroom activity. We wanted to know 
what was happening at a given time to permit us to examine, 
relations among co-occurring aspects of classroom activities, 
with particular focus on instruction associated with use of 
technology resources. For example, if teachers were lectur- 
ing, how was the class organized and what resources were in 
use to support the lecture? When computers were in use, how 
challenging were the students' tasks, how were they working 
with the teacher and one another, and what proportion of them 
were engaged in productive interaction? If students were 
composing long texts or projects, what resources were they 
using to support their work, and what symbol systems were in 
use? 

Our Tool 

We developed a versatile, technology-based observation 
tool that could serve our needs for research and evaluation. 
The tool also gives education professionals an easily learned 
observation method that permits rapid analysis and display of 
results. In the observation scheme that resulted, instruc- 
tional activities are the central organizing blocks. The 



emphasis of the scheme is on capturing the nature of instruc- 
tional tasks, the roles of teachers, the nature of social 
relationships, the variety of resources, and the responses of 
students. The instrument's breadth of coverage is coordi- 
nated w.^.h qualitative techniques for collecting fine-grained 
descriptions of instructional content and process. 

The instrument uses a time-sampling procedure to mini- 
mize rater bias, which is likely to be greater if raters 
sample "events" (since events are difficult to define when so 
many instructional characteristics are coded) , or if raters 
make summary judgments over an entire class period. The 
observation form is therefore organized for recording in 
timed intervals , The form is machine scannable, which 
permits instant updating and rapid analysis of the resulting 
database . 

Observers code 3 (a) a few key indexing variables once for 
the activity period to be observed (subject area, number of 
students assigned, classroom organization) , and (b) a set of 
activity descriptors on a time sampling schedule throughout 
the period. The following are the indexing variables: 

•Subject Area (one set for elementary and another set for 
secondary level) 

•Number of Students Assigned to the observed activity 

The activity descriptors are then coded once during each 
5-minute (elementary) or 10-minute (secondary) interval. 



3 A coding manual is available from the authors. 
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During an observation period, our coders observe for 1 minute 
% at the start of each 5~ or 10-minute interval; they then 

code just what occurred within that prior minute and use any 
remaining time to record field notes. The activity descrip- 
9 tors include: 

•Classroom Organization: teachtr-ied, independent work, 
group/cooperative, group/collaborative (jointly 
Q produced product) , pair/cooperative, 

pair/collaborative, pair/tutoring, student-led 
•Adult Roles : 

• Directing Instruction (codes that apply only to 

teacher-led classroom organizations) : 
explain/provide information, question (for compre- 
9 hension or examination), answer students' questions, 

direct students' work (step by step), correct/grade, 
test, read to students 
% Facilitating instruction (codes that apply to indepen- 

dent, cooperative, and collaborative work) : moni- 
tor/rove to help students at work, f -'Militate 
% discussion, conference, joint probi-,a- solve 

Management and Discipline: manage, discipline 
Not Present (with the group currently observed) 
0 •Symbol Systems serving key instructional functions in 

the material the teachers make available to students: 
verbal, numeric, math symbols, graphic, chart, diagram, 
£ pictorial, model, map, puzzle/pattern, motcr/action, 

music, objects 
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Symbol Systems students use in their products: verbal, 
numeric, math symbols, graphic, chart, diagram, picto- 
rial, model, map, puzzle/pattern, motor/action, music, 
objects 

Length of the Responses expected of students: 
repeat /copy (student replicates provided material 
exactly-e.g. spelling practice, cursive practice, key- 
boarding drill) , select (multiple choice, true/false) , 
short (no more than a sentence in length) , medium (no 
more than a paragraph in length) , long (multi-paragraph) 
•Level of Processing expected of students: low (emphasis 
on rote recall), medium (requiring inference or problem 
solution within a well-structured problem context),, high 
(requiring inference and construction of a response in a 
less structured task context) 
^Resources in Use 

Textual, including textbooks (textbooks, assigned 
literature, workbooks/worksheets, tests), print 
resources (library books, reference books, periodi- 
cals, reference/help sheets), materials (paper, 
file cards, blackboard), student's own work 
Hands-on materials 

Computer, including instructional software (electronic 
worksheet, simulation/ strategy) , and applications 
(word processing, HyperCard, graphics, database, 
spreadsheet, programming, telecommunications) 
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Other technology: laserdisc, scanner, film/video, 
slide/filmstrip, audio, robotics, class monitor, 
overhead, MIDI, calculator 
•Students' Responses to the Activities: 

Appropriateness of students' behavior 
Students' focus and investment 
Productive student-student interaction 

Results of the time-sampled observations are scanned, 
analyzed, and displayed in graphic or tabular formats on a 
Mac II. Currently the resultant displays are catalogued for 
flexible retrieval using a menu-driven interface. A goal is 
to develop a user-friendly interface for real-time, on-line 
queries . 

Illustra tive rises 
The instrument provides us with "snapshot" descriptions 
of classroom instruction which can serve multiple research 
and development functions. In this paper we illustrate two 
of our current uses: documentation of commonly reported 
changes in high access classrooms and model-driven descrip- 
tions of change. 

Documentation Of Reported Changes I n Hiyh Access Classrooms 

There is a clear need for empirical documentation of 
commonly reported changes in classroom practices in high 
access environments. Researchers and educators have made 
informal observations that computer use is associated with: 
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more challenging projects, less directed teaching and more 
teacher facilitation, more frequent group projects, more time 
on task, and more peer assistance (see, for example, Collins, 
in press; Hawkins & Sheingold, 1986) . However, there is 
little empirical documentation of these changes. 

Observations collected in 1989-90 have provided us with 
enough data to make clear the importance of careful investi- 
gation. Our analyses to date indicate certain associations 
between technology use and classroom activities that are 
consistent with informal reports. But some of our results 
are not consistent with what now appear to be overly general 
and overly romantic sketches of technology's impact. Our 
data suggest that the teachers make motivated choices about 
resources and pedagogical methods based on subject area; thus 
their classroom activities are not technology-driven in any 
simple way. Our findings serve to underscore the importance 
of documenting technology impact (a) within specific subject 
areas, (b) at particular levels of schooling (e.g., elemen- 
tary vs. secondary), and (c) for specific uses of technology. 

We illustrate our analyses of associations between tech- 
nology use and classroom instruction with 1989-90 observa- 
tions collected at our elementary site in language arts and 
in mathematics. The database included 144 5-minute intervals 
(12 hours) in language arts and 73 5-minute intervals (6.1 
hours) in mathematics. These data represent a less than 
adequate sampling of teachers' instruction, although observa- 
tions were made of activities that observers and teachers 
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agreed were representative of each teachers' instruction. We 
must stress the illustrative nature of our results. 

The analysis strategy was based on a nested series of 
queries. We defined a set of resource contexts, beginning 
with a simple distinction between those where the computer 
was in use and those where it was not, and compared instruc- 
tional patterns in those two contexts. We then made further 
refinements as questions for analysis emerged; for example, 
how did instruction differ when computers were used for 
applications (such as word processing) versus instructional 
software? Most analyses of instructional patterns were based 
on aggregations of individual codes. 4 For example, when we 
examined instruction and support roles we created two summary 
categories: Directing Instruction and Facilitating 
Instruction, representing the use of any of the roles in 
those two categories respectively. 

I ll ustrative results Language Arts anH tn athamaH a j- 

elementary al£&. 

The results for language arts were most consistent with 
informal reports of instructional patterns in high access 
contexts. In language arts, computers were in use a bit less 
than 20% of the time (Figure 1), and uses were always for 
applications rather than for delivery of instructional soft- 
ware. Consistent with informal observations of classroom 



Descriptions of procedures for aggregation are available from the authors. 
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organizations and teacher roles during student computer work, 
use of applications-primarily word processing-was associated 
with independent student work rather than teacher-led work 
(Figure 2), and with a role for teachers as facilitator of 
students' work rather than as director and deliverer of 
information (Figure 3) . 

In mathematics, while computers were observed in use 
slightly more frequently (32%) than in language arts (Figure 
4), the more striking difference was in type of use. 
Computers were used most often for instructional software 
rather than for applications. While we found relationships 
between computer use and classroom organization (Figure 5) 
and teacher role (Figure 6) that were similar to language 
arts, the similarity to language arts did not hoxd when we 
examined particular computer uses. When we compared use of 
instructional software to use of applications in mathematics, 
students were more likely to be working independently or 
cooperatively (rather than under the direction of the 
teacher) , and teachers were more likely to be facilitating 
(rather than directing) students' work only when students 
were using instructional software (Figures 7 and 8) . The 
small number of observations of application use limits infer- 
ence, but the difference in instructional software versus 
application use does make clear the importance of examining 
instructional patterns in terms of the specific functions 
that technology serves within instruction of particular 
subject areas. 
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Adult Role Across Resource Contexts Language Arts 
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Computer Use by Subject - Mathematics 
Percentage of intervals in which each variable was observed 
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Reports that technology use supports more challenging 
student work were supported only by our pilot data for 
language arts : When computers were in use (when students 
were using word processing for writing), tasks were somewhat 
more challenging (Figure 9) and longer in length (Figure 10) . 
In contrast, mathematics tasks were somewhat less challenging 
(Figure 11) and shorter in length (Figure 12) on computer; 
students were using instructional software to practice basic 
mathematics skills. 

In summary, instructional patterns observed in these 
high access classrooms differed both by subject area and by 
the nature of computer use. 

Model-Driven Descriptions of instmrt ionai r.han^ 

Our instrument is designed to provide documentation of 
instructional practices associated with technology use and 
changes in instructional practices over time. The value of 
the descriptions we produce is markedly enhanced when 
descriptions are guided by a mciai of instructional change. 
If patterns change over time as predicted by a model, our 
results provide validation for the model; if patterns are 
inconsistent with a model of change, our results suggest 
needed revisions in the mcdel. 

In this section, our examples illustrate an approach we 
are taking to model-driven methods of data analysis. We have 
drawn from two frameworks to help us articulate our expecta- 
tions for instructional impact of high technology access. 
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Level of Processing Across Resource Contexts - Language Arts 
Percentage of intervals within each resource context that each variable was observed 
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Level of Processing Across Resource Contexts - Mathematics 
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Toward a model Qf instructio nal change: Two V ev frame- 
work 8 . David Dwyer, Cathy Ringstaff, and Judy Sandholz 
(Dwyer, Ringstaff, & Sandholz, 1990) have proposed a model of 
ACOT teacher change based on analyses of ACOT teachers' regu- 
larly dictated audiotape records of their ACOT experience. 
Dwyer and his colleagues propose a five-phase process of 
instructional change: 

•Entry: the technology is implemented and a team of 

teachers selected 
•Adoption: basic instructional patterns are main- 
tained, with technology support for drill, practice, 
and word processing 
•Adaptation: teachers find that their instructional 
program is completed more rapidly and efficiently, 
freeing time for exploration of new curricula and 
pedagogy 

•Appropriation: computer expertise enables experimen- 
tation 

•Invention: teachers invent and implement fundamen- 
tally new forms of learning and teaching 

These phases of instructional change can be interpreted 
as phases in which "technology push" leads to a succession of 
newly emerging instructional goals (cf. Baker, 1988). The 
phase descriptions do not point consistently to particular 
causes or contexts of change, however. For example, 
"efficiency" is cited in the transition from adoption to 
adaptation, but it is not clear how teachers recognize it or 
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choose then to depart from traditional practices. Nor is a 
particular model of "new forms of learning and teaching" 
proposed. 

To provide that model-and thus a model for the final 
"invention" phase in Dwyer et al.'s (1990) framework-we have 
adapted the analysis of "inquiry environments" proposed by 
Marlene Scardamalia and Carl Bereiter (Scardamalia & 
Bereiter, in press; Scardamalia, Bereiter, McLean, Swallow, 
& Woodruff, 1989) . In their analysis of visionary, technol- 
ogy-supported instructional environments, technology is a 
valued resource that can support (a) "learning goals" rather 
than "task goals" and, (b) depth and breadth of knowledge 
construction among students and teachers rather than simply 
information delivery and retention. Adapting the constructs 
contained in our observation instrument (Table 1) , we charac- 
terized activities in these classrooms as follows. Projects 
are organized across disciplines which share corresponding or 
interconnected core concepts. Resources permit active 
construction of understandings via multiple and flexible 
representations of content. Students' work requires consid- 
erable initiative and construction of understandings; 
students' compositions are often lengthy. Learners use a 
variety of resources among themselves and the help of 
supportive adults. Students are often working cooperatively 
and teachers are often facilitating rather than delivering or 
directing instruction. 
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Table 1 . Observation categories applied to final Invention phase in Dwyer et al.'s model (1990) 



OBSERVATION 


INVENTION 
learning as Inquiry 
knowledge- 

building 

community 


Subject areas 


Integrated 
Interdisciplinary 


Resources 


Multimedia 
Interactive 

Multi-representational 


Task 

characteristics 


Technology- 
supported activities 
requiring considerable 
student planning, 
Inference, integration, 
application, and 
construction 


Classroom 
organization 


Common use of 
technology- 
supported 
cooperation and 
collaboration 


Instruction and 
support roles 


Common uses of a 
range of facilitating 
roles, Including 
conferencing and joint 
problem-solving 
Use of supportive 
telecommunications 
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How might our instrument provide descriptions of 
instruction that could validate-or suggest revisions-in Dwyer 
et al.'s phases of instructional change? Table 2 is an 
outline of instructional characteristics that can be docu- 
mented with our instrument and that fit three illustrative 
phases of Dwyer 1 s model, including a Scardamalia & Bereiter- 
like interpretation of Dwyer "s "invention" phase. 

Illustrative results . Comparisons between language arts 
instruction at our elementary site and English instruction at 
our secondary site illustrate how inferences can be made 
regarding the "fit" of our observations to various stages. 
The database consisted of 145 5-minute intervals at the 
elementary level (12 hours) and 45 10-minute intervals at the 
secondary level (7.5 hours). Again we must stress that the 
results represent illustrations of approaches to analysis, 
not results that we necessarily expect to remain with more 
extensive data collection. The patterns we found suggested an 
association between school level and degree of instructional 
innovation. Compared with the secondary teachers, the 
elementary level teachers in our samples appeared to be 
considerably further from the visionary model of inventive, 
instructional inquiry environments sketched above. 

Subject area. At the elementary school level, only one 
core sub ject— science— was ever double-coded with language 
arts. Field notes indicate that students were engaged in 
science writing (Figure 13) . In contrast, at the secondary 
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Table 2 (^T^fm° Ttes applied 10 three proposed phases 01 jns,ructional cha "9e 



OBSERVATION 
Subject areas 

Resources 



Task 
characteristics 



Classroom 
organization 



Instruction and 
support roles 



Proposed phases of Instructional change 



ADOPTION 
traditional 

instruction 
Instructional 

software 
word processing 

isoiaTed" 



Traditionalrtexts 
Instructional software 
Word processing 



Structured 
Basic skills.inference 
in well structured con- 
texts, brief answers 



Teacher-led 
Independent 



Directing 
Facilitating 



APPROPRIATION INVENTIOTT 

computer exper- learning as Inquiry 
tlse enables exper- knowledge- 

Imentatlon with building 

curriculum and community 
pedagogy 

Experiments in Integrated" 

technology-supported Interdisciplinary 
integration 



Declining reliance on Multimedia 

texts and Interactive 

instructional software 
Word processing 
Experiments in 

multiple media and 

representations 



Multi-representational 



Structured 

Experiments with 
some technology- 
supported activities 
requiring higher-level 
reasoning and 
student construction 

Teacher-led 

Independent 

Experiments 
in cooperation and 
collaboration (some 
technology- 
supported) 



Technology-supporte< 
activities requiring 
considerable student 
planning, inference, 
integration, application! 
and construction 



Common use of 
technology- 
supported 
cooperation and 
collaboration 



Directing Common uses of a 

Facilitating, with range of facilitating 
experiments in roles, including , 
specific techniques conferencing and joint 
such as conferencing problem-solving 

Use of supportive 
telecommunications 
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Double-coding With Language Arts Within Each Subject Area - Elementary Level 

Percentage of periods in which Language Arts was double-coded with each subject 
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Illustrative Findings 

Wednesday September 05, 1990 11:02.12 AM 
Figure 13 
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site, English was judged as integrated with another subject- 
social studies-70% of the time (Figure 14) . 

Resources in use. At the elementary level, the 
resources in use tended to be textual and not technological. 
Computers were in use slightly less than 20% of the time 
(Figure 15) . Computer uses, however, were exclusively appli- 
cations rather than instructional software; constructive 
writing tools-word processing-were the applications in use 
(Figure 16) . There were no observations of software provid- 
ing other forms of representation (e.g., graphics) or multi- 
representational technologies such as laserdisc, video or 
audio (Figure 17) . Since children were in fact engaged in 
art and music activities associated with their language arts 
curriculum, our findings revealed that the teachers were not 
yet exploiting the potential of technology to support these 
same activities . 

At the secondary site, the resources in use were also 
more often textual than technological. Computers were in use 
here about 35% of the time (Figure 18), again exclusively for 
applications rather than for presentation of instructional 
software. There was some variety in type of applications- 
word processing, HyperCard, and graphics (Figure 19) . In 
addition, occasional use of interfacing multi-representa- 
tional technologies was noted-audio and scanners (Figure 20) . 
Thus at the secondary site we did observe some technology- 
based tools for multi-representational activities. 
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Double-coding With English Within Each Subject Area - Secondary Level 
Percentage of periods in which English was double-coded with each subject 
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Figure 14 
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Number of Intervals Observed Within Each Resource Context - Elementary Level 

Count of intervals in each context 

Legend 




Periods 



Ch On Computer 

Of Off Computer 

In Instructional Software 

Ap Applications 




Ch 



a 



In 



Ap 



Illustrative Findings 

Tuesday September 04, 1990 1:24.27 PM 
Figure 15 
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Number of Intervals in which Computer Applications were Observed - Elementary Level 

Total number of intervals observed and number of intervals in which each application was observed 
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Illustrative Findings 

Wednesday September 05, 1990 11:17.55 AM 
Figure 16 
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Percentage of Intervals in which HypcrMedia were Observed - Elementary Level 

Percentages of total intervals in which computer applications were observed with other technologies 
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Illustrative Findings 

Tuesday September 04, 1990 1:20.48 PM 
Figure 17 
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Number of Intervals Observed Within Each Resource Context 

Count of intervals in each context 
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Tuesday September 04, 1990 1:20.53 PM 
Figure 18 
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Number of Intervals in which Computer Applications were Observed - Secondary Level 

Total number of intervals observed and number of intervals in which each application was observed 
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Figure 19 
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Percentage of Intervals in which HyperMedia were Observed - Secondary Level 

Percentages of total intervals in which computer applications were observed with other technologies 
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Figure 20 
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Nature of students' work. At the elementary level, the 
language arts tasks were rated predominantly as either medium or 
low in level of processing-thus, well structured activities with 
teacher-defined criteria for completion (Figure 21) . While 
teachers were utilizing word processing as support for students' 
writing, activities in which students participated substantively 
(in planning and coordinating the work) were rarely observed. 
Similarly, tasks were generally either short or medium in length 
and while task length tended to be longer with computer support, 
it was rarely judged as long in any resource context (Figure 
22) . 

At the secondary level, while well-structured activities 
with teacher defined criteria for completion were most 
common, ill-structured activities (high) were not uncommon 
(Figure 23) . The very low frequency of any code "off-comput- 
ers" reflects the finding that students in the secondary 
classrooms were rarely producing any assigned product without 
computer support. 5 Similar to the results for Level, tasks 
were rated at all possible lengths, including "long" (Figure 
24) . (These indices total more than 100% because multiple- 
coding within activity is permitted.) 

Classroom organization . At the elementary level in 
language arts, classrooms were generally organized for inde- 
pendent work, and teachers utilized computers heavily as 
support for independent work (Figure 25) . Cooperative work 



5 When students are engaged in listening, reading, watching, or taking notes, we do not code 
their participation for 'Length' or level of challenge'. 
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Level of Processing Across Resource Contexts - Elementary Level 

Percentage of intervals within each resource context that each variable was observed 
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Figure 22 
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Level of Processing Across Resource Contexts - Secondary Level 

Percentage of intervals within each resource context that each variable was observed 
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Illustrative Findings 

Tuesday September 04, 1990 4:28.05 PM 
Figure 23 
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Product Item Length Across Resource Contexts - Secondary Level 

Percentage of intervals within each resource context that each variable was observed 



Legend 




■ Repeat/Copy 
M Select 

§ Construct - Short 
Construct - Medium 
Construct - Long 



Ch On Computer 
Of Off Computer 



Ch 



a 



Illustrative Findings 

Tuesday September 04, 1990 4:28.56 PM 
Figure 24 
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Classroom Organization Across Resource Contexts - Elementary Level 

Percentage of intervals within each resource context that each variable was observed 
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Illustrative Findings 

Wednesday September 05, 1990 11:29.03 AM 
Figure 25 
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was very rare, and although truly collaborative projects were 
observed (jointly-produced products), these activities were 
not technology- supported (Figure 2 6) . At the secondary level 
in English, classrooms were generally organized for indepen- 
dent work with computer use, and teacher-led instruction off- 
computer (Figure 27) , Although cooperative activities were 
not uncommon on-computer, none of these was a collaborative 
activity (Figure 28) . 

Instruction and support roles. At both the elementary 
and the secondary levels, teachers were predominantly facili- 
tating instruction when students were on-computer and direct- 
ing instruction when off-computer (Figures 2 9 and 30) . 
Students were engaged in productive peer interaction more 
often with computer support (Figures 31 and 32) . 

Interpretation; A role for subject matter expertise in 
teaching. Why might secondary teachers be more able to 
create opportunities for technology-supported, constructive 
student work? To provide a possible explanation for the 
results, and therefore a possible explanation for instruc- 
tional change as described by a model like Dwyer et al.'s 
phase model, we return again to work of Scardamalia and 
Bereiter . 

Scardamalia and Bereiter (in press) argue that students 
engaged in constructive inquiry must be provided with 
resources representing multiple kinds of expertise. The 
kinds of expertise articulated-subject matter, curriculum, 
and pedagogical-can be distributed among teachers, students 
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Classroom Organization Across Resource Contexts - Elementary Level 

Percentage of intervals within each resource context that each variable was observed 
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Illustrative Findings 

Wednesday September 05, 1990 11:31.56 AM 
Figure 26 
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Classroom Organization Across Resource Contexts - Secondary Level 

Percentage of intervals within each resource context that each variable was observed 
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Illustrative Findings 

Wednesday September 05, 1990 11:40.43 AM 
Figure 27 
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Classroom Organization Across Resource Contexts - Secondary Level 
Percentage of intervals within each resource context that each variable was observed 
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Wednesday September 05, 1990 11:41.08 AM 
Figure 28 
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Adult Role Across Resource Contexts - Elementary Level 

Percentage of intervals within each resource context that each variable was observed 
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Figure 29 



9 

ERIC 



82 



83 



• 



0.8" 



0.6- 



0.4" 



0.2 ■ 



Adult Role Across Resource Contexts - Secondary Level 

Percentage of intervals within each resource context that each variable was observed 
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Task-related Student Interaction Across Resource Contexts - Elementary Level 

Percent of students exhibiting task-related interaction within each resource context 
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Figure 31 
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Task-related Student Interaction Across Resource Contexts - Secondary Level 

Percent of students exhibiting task-related interaction within each resource context 
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Figure 32 
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themselves, and instructional materials. Thus teachers are 

• not seen as solely responsible for providing expertise, but 
as contributors to the design of instructional environments; 
their expertise certainly helps them to know what is needed 

• to support a given project. Moreover, teachers' own engage- 
ment in building personal scholarship-subject matter exper- 
tise-provides a model to students of knowledge-building 

• activities. 

Scardamalia and Bereiter do not address directly how 
kinds of expertise can support constructive uses of technol- 

• ogy. It is reasonable to assume, however, based on their 
arguments, that understandings of a subject's concepts and 
methods enable teachers to envision how technology might 

• support inquiry within that discipline, if so, then the 
differences we found in technology use between the elementary 
and secondary level teachers are not surprising given typical 

i differences in subject area training and specialization for 

teachers at each level. Elementary teachers are curricular 
and pedagogical generalists within a tradition where curricu- 

I lum has been defined as a set of discrete facts and concepts 

not typically based on disciplinary expertise. The secondary 
teachers are likely to have somewhat greater subject matter 

I knowledge by virtue of the training required of them for 

certification. Secondary teachers also focus their curricu- 
lum development efforts within one subject area. It is 

> likely, then, that subject matter expertise, together with an 

instructional focus within one subject area, supported ACOT 
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secondary teachers' appropriation of technology's capabili- 
ties to support knowledge building and inquiry. 

Both sets of teachers, however, have yet to exploit the 
full potential of technology for fostering deep understanding 
of subject matter content. There was evidence of some 
instructional innovation at the secondary level, but the 
patterns tended to suggest local experimentation 
(adaptation/appropriation) rather than comprehensive revision 
(invention) . Further work is needed to understand how 
subject matter expertise, among a range of other factors, 
plays a role in te.ichers' construction of new conceptions of 
instructional environments. 

Conclusions and Fnt-nr** pirp^fion c 
Our goal is to develop tools that enable us to document 
the impact of technology on classroom instruction and on 
student, teacher, and parent outcomes. In this paper, we 
described one new tool-our new classroom observation instru- 
ment-and w,{ demonstrated its value for documenting instruc- 
tion. We provided two illustrations of its potential uses: 
documenting commonly-reported observations of instruction in 
high access classrooms, and validating models of the role of 
technology use on instructional change. 

The usefulness of our instrument will ultimately depend 
on both coordination with other kinds of qualitative data 
gathering and validation of the instrument through planned 
contrasts, either with classrooms utilizing technology in 
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